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Advancing the research agenda 
for diagnostic error reduction 

Laura Zwaan, 1 Gordon D Schiff, 2,3 Hardeep Singh 4,5 



ABSTRACT 

Diagnostic errors remain an underemphasised 
and understudied area of patient safety research. 
We briefly summarise the methods that have 
been used to conduct research on epidemiology, 
contributing factors and interventions related to 
diagnostic error and outline directions for future 
research. Research methods that have studied 
epidemiology of diagnostic error provide some 
estimate on diagnostic error rates. However, 
there appears to be a large variability in the 
reported rates due to the heterogeneity of 
definitions and study methods used. Thus, future 
methods should focus on obtaining more precise 
estimates in different settings of care. This would 
lay the foundation for measuring error rates over 
time to evaluate improvements. Research 
methods have studied contributing factors for 
diagnostic error in both naturalistic and 
experimental settings. Both approaches have 
revealed important and complementary 
information. Newer conceptual models from 
outside healthcare are needed to advance the 
depth and rigour of analysis of systems and 
cognitive insights of causes of error. While the 
literature has suggested many potentially fruitful 
interventions for reducing diagnostic errors, most 
have not been systematically evaluated and/or 
widely implemented in practice. Research is 
needed to study promising intervention areas 
such as enhanced patient involvement in 
diagnosis, improving diagnosis through the use 
of electronic tools and identification and 
reduction of specific diagnostic process 'pitfalls' 
(eg, failure to conduct appropriate diagnostic 
evaluation of a breast lump after a 'normal' 
mammogram). The last decade of research on 
diagnostic error has made promising steps and 
laid a foundation for more rigorous methods to 
advance the field. 

Diagnosis is one of the most important 
tasks a physician performs and deter- 
mines subsequent treatment and patient 
outcomes. Although most patients can 
expect to be diagnosed correctly, diagnos- 
tic errors are being increasingly 



demonstrated in the patient safety litera- 
ture. 1-5 Recent research shows that diag- 
nostic errors can often lead to severe 
consequences and preventable morbidity 
and mortality. 2 6 7 Despite an urgency to 
study and reduce diagnostic errors, for 
various reasons discussed in other articles 
in the supplement, they have remained 
an underemphasised and understudied 
area of patient safety research. 8 9 

Over the past 5 years, the annual 
Diagnostic Error in Medicine (DEM) 
conferences have highlighted emerging 
research in this area, which we summarise 
here. For the purposes of this paper, we 
classify current research in this field into 
three main topic areas: (1) epidemiology 
of diagnostic error (frequency, types, 
detection methods); (2) causes of diag- 
nostic error (cognitive and system issues) 
and (3) error prevention strategies (devel- 
opment, implementation and evaluation 
of interventions). This paper briefly sum- 
marises the methods that have been used 
to conduct research in these three areas, 
then outlines future research needs to 
better understand and reduce diagnostic 
error in medicine. 

Research methods to study the 
epidemiology of diagnostic error 

Existing research methods 

Multiple research methods, largely retro- 
spective, have been used to examine the 
epidemiology of diagnostic error (for an 
extensive overview on incidence, see 
Graber's article in this issue) 10 
Commonly used data sources include 
reports of malpractice claims, 6 11 chart 
reviews of selected diagnoses or hospital 
admissions 2 12 and autopsy reports. 13 14 
While these studies have given us a 
general appreciation of the burden of the 
problem, these estimates of incidence of 
the problem are far from precise or gen- 
eralisable. Retrospective methods also 
introduce hindsight bias in judgments 
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about error determination. Nevertheless, one advan- 
tage of retrospective methods is the potential availabil- 
ity of longitudinal data that spans the continuum of 
care of the patient (ie, outpatient visits, multiple sub- 
sequent hospital admissions, outcomes, etc). This 
allows researchers to retrospectively track the evolu- 
tion of the diagnostic process over time and link it to 
diagnostic outcomes (ie, final diagnosis). 2 

Prospective methods have advantages such as reduced 
hindsight bias but used infrequently to study the epi- 
demiology of diagnostic error. One example is the use 
of 'standardised patients' who present to providers in 
routine clinical settings without revealing their actual 
purpose. 15 16 One study involved 23 rheumatologists 
who were visited by a standardised patient with known 
psoriatic arthritis. The diagnosis was missed or wrong in 
nine visits (39%). However, such methods are resource 
intensive. Other types of prospective methods involve 
presentation of hypothetical cases to assess physician 
cognitive error rates. 17 18 However, these methods are 
mainly used to determine how varying circumstances or 
characteristics of physicians or patients influence error 
rates, and do not help determine incidence rates repre- 
sentative of a certain population. 

Although previous studies have given us a general 
appreciation of the burden of diagnostic error, the 
estimated incidence rates of diagnostic errors vary 
greatly, from about 1% to as high as 55% in certain 
diseases or patient groups. 1 This large variation can in 
part be explained by four factors: 

1. Setting of the error and patient population: errors in 
certain specialty or ancillary care settings, such as radi- 
ology, appear to be less common than errors in clinical 
medicine. Also, diagnostic errors are more likely to 
occur with older or more complex patients compared 
with younger patients who have fewer comorbidities. 1 

2. Study methods: higher error rates appear to be related in 
part to use of data sources that over-represent severe or 
lethal outcomes, such as autopsy studies and malpractice 
claims. Conversely, other data sources, such as chart 
reviews, might not uncover all diagnostic errors due to 
lack of documented information (ie, ascertainment bias). 

3. Type of diagnosis: certain diagnoses are likely more 
easily missed or confused, especially at an earlier stage of 
the illness, for example, tuberculosis or HIV 

4. Definition of diagnostic error: some studies define diag- 
nostic error as any diagnosis that is wrong, delayed or 
missed, regardless of whether an error in the diagnostic 
process occurred. 19 20 Other studies define diagnostic 
errors only when there is clear evidence of 'missed 
opportunities' for earlier diagnosis in a patient's 
care. 12 21 Therefore, the rates of diagnostic error found 
in different studies are likely to vary substantially accord- 
ing to criteria used to define 'error'. 

Directions for future research on the epidemiology of diagnostic error 
Obtaining more precise rates of error is an enormous 
challenge, given the heterogeneous data and 



methodological and logistical challenges noted above. 
Nevertheless, such an undertaking is necessary to 
establish base rates in common settings of care 
(primary care, general medical hospitals, etc), to both 
attract needed attention to the urgency and magnitude 
of the problem as well as prioritise and evaluate inter- 
vention efforts. Such a project could have an import- 
ant impact in ways similar to the IOM's landmark 
report on medical errors and patient safety, 22 which 
propelled the field of patient safety forward. 
Suggestions for specific methodological and research 
approaches to advance this agenda include: 

1. Synthesise existing studies to estimate rates of diagnostic 
error: the true incidence of diagnostic error remains 
unknown. As more sophisticated methods for detecting 
diagnostic error are being developed, an early first step 
would be to apply rigorous methods to synthesise find- 
ings across studies in order to estimate incidence rates of 
diagnostic error. We recommend that incidence rates of 
diagnostic errors should be determined separately for the 
different settings (eg, inpatient hospital setting, primary 
care and outpatient setting) to account for differences in 
patient populations and diagnostic techniques. A synthe- 
sis of existing studies would also help further clarify the 
effect of methodological differences among the studies. 

2. Evaluate incidence rates over time: longitudinal studies are 
needed to determine trends related to the incidence of 
diagnostic error, to identify any fluctuations in types of 
diagnostic errors in different settings and to understand 
what types of diagnoses might be more or less vulnerable 
to diagnostic error over time (eg, in response to the adop- 
tion of new diagnostic procedures and technologies). In 
addition, within organisations and systems that are imple- 
menting strategies to reduce error, it is important to know 
whether general patient safety interventions (such as 
improved safety culture, patient engagement), or certain 
targeted interventions (such as techniques to improve test 
result follow-up) actually result in lower rates of diagnostic 
error. Conducting studies using similar definitions, meas- 
urement methods and time intervals (eg, every 5 years) can 
provide insights into which processes, settings, patient 
groups or diseases are in greatest need of attention over 
time, and which interventions appear to have the greatest 
impact on patient safety. 

3 . The use of Electronic Health Records (EHRs) to identify 
diagnostic error: currently, manual chart reviews are con- 
sidered the gold standard for detecting diagnostic errors. 
This method is labour-intensive and extensive resources 
are required. 23 Several recent studies illustrate the use of 
EHR-based algorithms to identify patient charts with a 
higher probability of a diagnostic error. 4 7 24 Such 
methods should now be explored further and refined for 
use in other institutions. Additional methods using more 
sophisticated algorithms should also be developed to 
proactively identify diagnostic errors before patient 
harm. This technique could also lend itself to study 
causes of error as well as facilitate tracking of error rates 
over time and testing the impact of interventions. 
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Causes of diagnostic error 

Currently used research methods 

Because many aspects of the diagnostic process are 
not easily captured using current study methods, the 
causes of diagnostic error are difficult to identify. 
Ideally, the diagnostic process is studied prospectively 
in a real clinical setting, but this poses substantial 
practical difficulties (eg, ethical issues, time con- 
straints, measuring clinician thought processes unob- 
trusively, observer bias). Consequently, most studies 
conducted in naturalistic settings have examined the 
causes of diagnostic error retrospectively (eg, through 
record review which sometimes is followed by inter- 
views). 2 20 21 25 These studies have revealed vulner- 
abilities in clinical workflow processes or 
organisational issues that contribute to error (eg, pol- 
icies that are ambiguous about who is responsible for 
test results follow-up). 26 They have also yielded illus- 
trative patterns of cognitive failures or biases, (ie, 
errors in the reasoning process of the physician), 1 as 
well as the complexity of interaction between system 
and provider factors. 25 Prospective evaluations have 
been generally restricted to artificial settings (eg, case 
vignette studies) 27 and offer a complementary cogni- 
tive perspective. 

Prior studies have revealed that the diagnostic step 
of 'information synthesis' was particularly prone to 
error. 1 4 19 20 Retrospective studies conducted in nat- 
uralistic settings can identify contributory factors and 
types of diagnostic breakdowns 7 ; however, they are 
seldom able to reveal the root causes of diagnostic 
error due to limited information about causality. For 
instance, in reviewing an error case in which a diagno- 
sis was missed despite the availability of laboratory 
results to support the diagnosis, it may be difficult or 
impossible to know whether the physician lacked the 
appropriate knowledge to arrive at a correct diagnosis, 
or was influenced by a cognitive bias, distraction or 
faulty heuristic, or simply did not have time or access 
to the result. 18 28 29 

Prospective studies in controlled settings could be 
potentially useful to examine causal relationships. 
Experimental designs, in particular, can isolate the 
effects of certain biases or the mechanisms underlying 
the reasoning process. 18 30 However, the next step for 
any such line of research would be to better under- 
stand how problems detected in the laboratory trans- 
late into real-world behaviours and problems. A key 
will be enabling a 'culture of safety' that will permit 
clinicians to honestly reflect on errors they have been 
involved in, and examine and test ways they could be 
prevented in their daily work in the future. 

Directions for future research on causes of diagnostic error 
1. Linking naturalistic and experimental studies: studies on 
the causes of error in naturalistic settings and experimen- 
tal (case vignette-based, simulation-based, etc.) clinical 
research studies each carry distinct advantages and 



limitations. To maximise the knowledge gained from 
these disparate methods, we believe that more can be 
done to encourage linkages between research paradigms. 
The results of naturalistic studies could inform the 
hypotheses of experimental studies (eg, to better under- 
stand causal factors), and in turn, findings from these 
experimental studies could be tested for their applicabil- 
ity and validity in a naturalistic setting. For example, fre- 
quently missed lesions in x-ray evaluations that have 
been found in studies on malpractice claims could be 
studied in a clinical research setting (eg, by studying the 
eye-movements of radiologists when reviewing the 
x-rays). By replicating these errors in the clinical research 
setting, we can study why they occur and how the 
process could be improved. 'Triangulating' methodo- 
logical approaches in this manner could accelerate our 
understanding of the pathogenesis and prevention of 
diagnostic error. 
2. Testing newer models to study diagnostic error: newer 
models can be used to integrate and align the objectives 
of naturalistic observational studies and experimental 
studies. Advanced techniques for 'root cause analysis' 
developed for accident investigation outside of medicine 
have barely begun to be used in healthcare and offer 
promising returns. Another potentially fruitful avenue of 
research is application of the 'situational awareness' 
model, which has been widely studied in the human 
factors literature, specifically in the field of aviation. 31 It 
can be applied to examine how practice environments 
and systems influence physicians' cognitive processes, 
helping or hindering their ability to differentiate infor- 
mation relevant to diagnosis while maintaining a broader 
general overview of the patient. 32 Situational awareness 
emphasises clinicians' understanding of 'what's going on' 
and 'what's likely to happen next' as key elements in 
effective diagnostic decision making. 33 

Improving diagnosis 

Current intervention research 

A recent systematic review evaluated the effects of 
patient safety strategies that focused on diagnostic 
error and found that only few strategies had an effect 
in terms of diagnostic error reduction or reduction of 
patient harm. 34 While this review identified over 100 
studies that tested the effects of interventions on the 
diagnostic process, two other recent narrative reviews 
showed that there are a large number of interventions 
that have been developed to reduce diagnostic errors, 
but have not been yet tested for their effectiveness. 
Specifically, these reviews evaluated the existing litera- 
ture on outcomes of system-related and cognitive 
interventions. 35 36 System-related interventions are 
those that are focused on addressing organisational 
process errors, and vulnerabilities, whereas cognitive 
interventions focus on improving physicians' percep- 
tual and thought processes (eg, educational interven- 
tions, cognitive de-biasing strategies; see related 
papers in supplement). Whereas many of the 
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suggested interventions were judged to have great 
potential, only few were actually tested and imple- 
mented in practice. This paucity of published 
outcome studies underscores the urgent need to not 
only conceptualise and develop interventions but to 
systematically evaluate their effectiveness. 

Directions for future research to improve diagnosis 

1. Patient engagement: strategies to promote shared decision 
making and patient engagement in healthcare hold great 
promise for reducing errors and error-related harm. 37-39 
The patient is key stakeholder at every step in the diagnos- 
tic process and serves as the main source of information 
during the diagnostic process. 40 The patient also has 
access to information that can help the physicians main- 
tain and regain situation awareness. For example, patients 
may support physicians by accurately informing them in a 
timely fashion about changes in their clinical situation. 
Patients are uniquely suited to detect and report errors 
that occur during their care processes. Thus patients are 
an enormous and virtually untapped resource for reducing 
diagnostic error; 41 ^ 3 research studies that examine this 
potential are few. Therefore, studies are warranted to 
examine ways in which patients can influence their physi- 
cians' decisions by providing more accurate or focused 
information, asking questions, questioning diagnoses and 
discussing medical tests and diagnoses they may be con- 
cerned about. 44 To be able to involve the patient in their 
medical process, the physician's role is changing as well. 
The physician needs to facilitate patient involvement by 
creating an environment in which patients can contribute, 
without shifting responsibility for accurate diagnosis to 
the patient. Specifically, the physician needs to create 
opportunities for patient involvement, for example, 
provide the possibility for patients to upload the patient 
history (eg, family history) online in advance as well as 
creating an atmosphere in which the patient feels comfort- 
able to ask questions. 

Furthermore, research should explore the potential 
diagnostic benefits of patients' critical review of their own 
medical record. 45 

2. EHRs: EHRs are now being developed and implemented 
in many countries, and the focus on improving the diag- 
nostic process through the use of EHRs is timely. 
Research has already shown that the EHRs can be used 
for detection of diagnostic error, s 7 and the next step is 
to apply these methods to the implementation and evalu- 
ation of interventions to improve the diagnostic process. 
Schiff and Bates created a conceptual model and made 
several suggestions for preventing or minimising diagnos- 
tic error with use of EHRs and other health information 
technologies. 46 Some of the most promising suggestions 
are: 

► Improving access to information. An EHR provides lon- 
gitudinal information about the patient's health status, 
including prior outpatient visits and hospitalisations. 
Although the ready availability of this information 
should facilitate the diagnostic process, research is 



needed to understand how best to facilitate display and 
use of this information to improve the diagnostic 
process. In particular, it will be necessary to identify and 
correct unintended consequences of EHR adoption, 
including 'information overload.' 47 Incorporation of 
tools in the EHR to facilitate diagnosis. Checklists have 
been shown to offer great potential for reducing errors 
in general 48 49 and can be applied to the diagnostic 
process. 50 Although there is limited evidence thus far to 
support the use of checklists for diagnostic error reduc- 
tion, 46 51 the EHR provides new opportunities to 
incorporate checklists into clinical practice to facilitate 
the diagnostic reasoning process. 
► Exploring newer forms of Health Information technol- 
ogy (HIT): New information technologies, such as the 
IBM supercomputer Watson, have potential to improve 
the diagnostic process 5 by using natural language pro- 
cessing. Natural language processing allows systematic 
analysis of human natural language into medical con- 
cepts, which eventually can result in a shortlist of differ- 
ential diagnoses. Although we are far from broad 
implementation of such techniques in clinical practice, 
research is needed to evaluate which techniques have the 
potential to reduce error. An extensive review of ways to 
reduce diagnostic error using HIT is described in the 
article of El-Kareh et al in this special issue. 
3. Delineation of specific diagnostic process pitfalls: 
research on the frequency and causes of errors could 
reveal specific areas of improvement that can be targeted 
through focused interventions. For example, a commonly 
seen error in malpractice claims for missed or delayed 
breast cancer diagnosis is failure to pursue the appropri- 
ate additional diagnostic evaluation for a woman with a 
breast lump after her diagnostic mammogram returns as 
'negative'. 6 Rather than recognising the imperfect sensi- 
tivity of this test in the face of a palpable mass and pro- 
ceeding to do the recommended next test (eg, 
ultrasound-guided aspiration cytology), physicians have 
on occasion instead reassured the patient about the 
'normal' mammogram and stopped the diagnostic 
workup. 53 Attributing rectal bleeding to 'hemorrhoids' in 
a patient at high risk for colorectal cancer, rather than 
conducting additional diagnostic testing to rule it out, 
illustrates another common diagnostic pitfall. 12 Specific 
pitfalls that contribute to greater-than-expected burden 
of diagnostic errors could be targeted through training 
and system redesign. 
In conclusion, research on diagnostic error, although 
still nascent, has evolved significantly as compared 
with the state of knowledge a decade ago. In this 
paper, we highlight areas where research methods 
could propel the field forward in promising directions 
related to epidemiology, contributory factors and 
interventions. The recently founded Society of 
Improving Diagnosis in Medicine and the annual 
Diagnostic Error in Medicine conferences are plat- 
forms to bring together a multidisciplinary group of 
people (including clinicians, patients, researchers and 
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educators) to advance the research agenda. This 
agenda could also be integrated with research agendas 
of other organisations (eg, National Institutes of 
Health, specialty societies as well as national research 
agendas, eg, the UK's large research agenda on 
cancer). Furthermore, funding in this area should be 
stimulated to operationalise this research agenda. 
Using well coordinated, multidisciplinary approaches, 
and with appropriate research support, the foundation 
laid thus far can lead to future methodological 
advancement in the field and to reduction of diagnos- 
tic errors. 
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